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1.0  INTRODUCTION 


The  Joint  Task  Force  on  the  So£tvare  Technology;  for  Adaptable, 
Reliable  Systems  (STARS)  Program  was  formed  at  the  direction  of  the 
Deputy; Under  Secretary:  of  Defense  for  Research  and  Engineering 
(Research  and  Advanced  Technology)  (DUSD(R&AT))  with  support  from  the 
Assistant  Secretary .of  the  Army:(RD&A),  the  Assistant  Secretary:  of 
the  Navy  (RE&S),  the  Assistant  Secretary: of  the  Air  Force  (RD&L)  and 
the  Deputy  Under  Secretary: of  Defense (C^I)  to  refine  the  strategy: for 
the  program,  to  prepare  for  detailed  planning  and  management  of  the 
prograa,  and  to  make  recommendations  concerning  a  draft  plan.  In 
existence  for  the  four  months  from  15  November  1983  until  15 : March 
1982,  the  Task  Force  began  from  the  Strategy  for  a.  DoD  Software  Ini¬ 
tiative!^  published  1  October  1982,  and  ended  by  producing  a  number 
of  documents  including  this  and  others  listed  in  Section  4‘.0. 

The  Task  Force  was  composed  of  two  or  three  representatives  from 
each  Service  and  members  from  the  Defense  Cotmaunications  Agency,  the 
National  Security  Agency ;  and  the  DoD  Computer  Security : Center.  The 
Chair  was  from  0DUSD(R&AT).  Several  general  consultants  and  a  number 
of  subject  area  specialist  consultants  supported  the  Task  Force's 
efforts.  The  Task  Force  members  and  consultants  are  listed  in  Sec¬ 
tion  3.0. 

The  Task  Force  arrived  at  recommendations  covering  the  technical 
tasks  that  should  be  undertaken,  how  the  STARS  program  should  be 
managed,  and  how  some  major  tasks  should  be  packaged  for  contracting 
purposes.  These  recoraendations  included  identifying  those  early: 
tasks  that  are  on  the  prograa's  critical  path(a)  and  must  therefore 
be  performed  expeditiously; 


^Department  of  Defense,  Strategy  for  DoD  Software  Initiative,  in  two 
Volumes,  ODUSD  (R&AT),  1  October  1982. 


The  next  section  reviews  the  Task  Forces  activities.  Section 
3.0  describes  the  membership  of  the  Task  Force  and  identifies  the 
consultants  involved.  Section  4i0  lists  the  documents  produced,  and 
Section  5i0  lists  recommendations  for  early: tasks  to  be  expedited. 
Finally;  Section  6.0  acknowledges  the  many  persons  who  have  assisted 
the  Task  Force's  efforts. 


2.0  TASK  FORCE  ACTIVITIES 

The  Task  Force  began  on  IS  November  1982  by. receiving  two  days 
of  briefings  covering  both  volumes  of  the  October  1  Strategy  for  a 
DoD  Software  Initiative,  the  Ada  Program,  a  preliminary : survey : of  DoD 
softvare  RSD,  and  the  Report  of  the  DoD  Joint  Service  Task  Force  on 
Software  Problems.  ^  In  November  and  early : December,'  the  Task  Force 
concentrated  on  three  items: 

o  Revising  the  1  October  strategy : document  based  on  comments 
from  the  Services. 

o  Improving  the  preliminary:  survey:  of  existing  or  already: 
planned  software  R&D  in  DoD. 

o  Producing  a  first  draft  of  a  Program  Management  Plan  to 
cover  many:  of  the  DoD  management  issues  that  had  been 
treated  only : lightly :  in  the  1  October  document  or  had 
received  comment  from  the  Services. 

In  mid-December,  subject  matter  consultants  for  each  of  the 
major  functional  task  areas  identified  in  the  1  October  strategy: 
document,  plus  the  Software  Engineering  Institute,  briefed  the  Task 
Force  on  their  outlines  for  more  detailed  strategies.  From  each  half 
day: discussion  the  consultant  for  the  functional  task  area  gained 
guidance  for  producing  a  draft  strategy  in  his/her  area.  These 
detailed  discussions,  plus  the  prior  efforts,  resulted  in  the  Task 
Force  members  rapidly:  becoming  thoroughly ; familiar  with  the  issues 
and  involved  in  their  resolution. 

The  Task  Force  dispersed  during  the  holidays  to  give  members  an 
opportunity:  to  discuss  progress  and  issues  with  their  home  organiza¬ 
tions.  In  January;  the  Task  Force  reassembled  to  receive  a  number  of 
briefings  from  interested  parties  inside  and  outside  DoD  and  to 
prepare  for  a  DoD  Software  Initiative  Workshop  to  be  held  February: 

^Department  of  Defense,  Report  of  the  DoD  Joint  Service  Task  Force  on 
Software  Problems.  ODUSD  (R&AT),  30  July: 1982. 
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7-9  in  Raleigh,  N. C.  Btiefings  were  received  on  selected  topics  from 
several  DoD  organizations  such  as  the  Ballistic  Missile  Defense 
Advanced  Technology:  Center  (Amy),  the  Naval  Research  Laboratory: 
(Navy),  and — after  the  workshop — the  Rome  Air  Development  Center  (Air 
Force).  From  outside  DoD  there  was  a  briefing  from  the  National 
Aeronautics  and  Space  Administration  (NASA)  as  well  as  a  number  of 
unsolicted  briefings  from  Defense  contractors.  A  memorandum  of 
agreement  with  NASA  was  drafted. 

Also  during  January;  draft  strategies  for  each  of  the  task  areas 
were  prepared  by:  the  consultants,  reviewed  by: the  Task  Force  and 
revised.  Preparations  were  made  for  the  initial  presentations  at  the 
workshop. 

To  aid  in  coordination  within  DoD,  a  Task  Force  member  briefed 
the  STARS  program  to  the  DoD  Software  Test  and  Evaluation  Project 
(STEP)  conference  February: 1-3  in  Washington,  D.C.  sponsored  by:  the 
National  Security  Industrial  Association  (NSIA)  in  cooperation  with 
DoD.  Several  additional  members  of  the  Task  Force  attended  this 
conference. 

A  Workshop  on  the  DoD  Software  Initiative  was  held  in  Raleigh, 
NArth  Carolina,  February  7-9.  The  purpose  of  the  workshop  was  to 
expose  the  draft  Functional  Task  Area  Strategies  widely:  across  the 
Defense  community . and  revise  them  as  appropriate.  Approximately  500 
persons  attended  the  workshop.  Of  these,  roughly:  300  were  Defense 
contractors,  15  0  DoD  personnel,  and  5  0  academic.  The  workshop  used 
parallel  panels  of  from  8  to  18  persons  for  each  functional  task 
area,  meeting  in  alternating  open  and  closed  sessions  to  review  and 
decide  on  revisions.  During  the  open  sessions,  the  Strategies  vere 
presented  and  persons  not  on  panels  had  a  chance  to  comment.  In 
addition,  many  persons  used  forms  provided  to  make  vritten  comments. 


The  vorkshop  resulted  in  improvements  to  all  the  Strategies. 
Most  changed  in  minor  or  moderate  ways  but  one,  Systems,  received 
sweeping  revision  and  another,  the  Software  Engineering  Institute, 
revealed  a  broad  range  of  diverse  opinions  on  its  functions  and  pro¬ 
posed  organization. 

In  the  workshop's  final  session,  reports  were  also  made  by:  two 
special  panels: 

o  A  group  of  senior  members  of  the  computing  community:  who 
anong  them  attended  all  the  open  sessions,  reported  on  their 
impressions  of  the  workshop  and  the  STARS  program.  Copies 
of  the  panel's  slides  are  shown  in  Appendix  1.3. 

o  A  group  of  senior  industry . attendees,  who  met  in  closed  ses¬ 
sion  throughout  the  workshop,  addressed  the  entire  program 
but  particularly : how  it  might  best  be  implemented.  A  copy: 
of  the  panel's  report  is  in  Appendix  1.4  ^ 

The  suggestions  of  these  panels  provided  input  particularly:  to  the 
preparation  of  the  STARS  Implementation  Approach. 

The  following  conclusions  were  drawn  from  the  workshop  and 
presented  to  the  attendees  of  the  closing  session. 

o  Ve  need  the  initiative  -  DoD  lead  is  proper 

o  Goal  is  appropriate  -  may: need  clarification 

«  Objectives  are  appropriate 

o  Attendees  liked  1  October  document  as  foundation  for  STARS 
o  Right  technology . issues  vere  identified 
o  Estimates  of  effort  vere  low 

o  Some  panels  got  lost  in  detail  -  top  level  plan  not  visible 


o  Structure  of  program  did  not  yet  provide  incentive  to  indus¬ 
try:  to  give  DoD  leverage 


o  Need  to  achieve  goals  through  integrated  projects 

o  Need  to  rethink  security  and  proprietary : software. 

Following  the  workshop,  the  Task  Force  concentrated  on  producing 
the  final  set  of  documents  (see  Section  410  for  a  complete  list). 
Revisions  were  made  to  the  Functional  Task  Area  Strategies,  reviewed 
by: the  Task  Force  and  the  workshop  panels  in  each  area,  and  edited  to 
final  form.  The  overall  strategy: and  program  management  documents 
received  revisions  and  an  implementation  approach  was  prepared. 

During  this  period  many : discussions  were  held,  including  all  day 
briefings  for,  and  discussions  with  the  Computer  Science  and  Technol¬ 
ogy  .Bbard  of  the  National  Research  Council,  National  Academy: of  Sci¬ 
ences  and  the  combined  Joint  Policy :  Coordinating  Group  for  Computer 
Resources  Management  of  the  Joint  Logistical  Commanders  and  the  Com¬ 
puter  Sciences  Subgroup  of  Joint  Directors  of  Laboratories.  One 
result  of  these  discussions  was  a  recommendation  for  the  establish¬ 
ment  of  a  panel  of  senior  people  to  further  investigate  and  recommend 
the  form  and  functions  for  the  Software  Engineering  Institute. 

The  Task  Force  members  attempted  to  establish  a  complete  base¬ 
line  of  current  and  planned  DoD  R&D  activity: in  the  functional  task 
areas.  While  the  information  achieved  was  very: useful  and  generally: 
sufficient  for  the  Task  Forces  purposes,  it  became  clear  that  the 
resulta  would  require  more  resources  than  were  available  to  ensure 
the  completeness  and  accuracy : required  for  publication. 
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3.0  MEMBERSHIP 

The  DoD  representatives  on  the  Task  Force  were 

Joseph  C.  Batz 
OUSDRE  (R&AT) 

Pentagon,  Room  3D1079 
Washington,  DC  20301 

Paul  M.  Cohen 

Defense  Communications  Agency 
DCEC  Code  R620 
1860  Wiehle  Ave. 

Reston,  VA  22090 

Samuel  A.  DiNitto,  Jr. 

RADC/COE 

Griffiss  AF3,  NT  13441 

Lt.  Col.  Larry  Druffel  (Chairman) 

ODSDRE  (R&AT) 

Pentagon,  Room  3E114 
Washington,  DC  20301 

Col.  Harold  C.  Falk 
AFWAL/AA 

Wright-Pat ter son  AFB,  OH  45433 
H.O.  Lubbes 

Naval  Electronic  Systems  Command 
Washington,  DC  20363 

Ann  M*rmor-Squirea 

DoD  Computer  Security  Center 

Fort  Meade,  MD  20755 

Carol  P.  Morgan 
NAVSEA-001 
JP2,  Room  188 
Washington,  DC  20362 

D.  Burton  Nevlin 
ODSDRE  (SAS  ) 

Pentagon,  Room  2A318 
Washington,  DC  20330 

Charles  Oglesby 
DARCOM,  DRCDE-SE 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 
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John  H.  Manley : 

Computing  Technology : Transition 
82  Concord  Drive 
Madison,  CT  06443 

Samuel  T.  Redvine,  Jr. 

MITRE 

1820  Dol ley  Madison 
McLean,  VA  22102 

Donald  Ji  Reifer 

President 

RCI 

2355  0  Hawthorne  Blvd.,  Suite  2  08 
Torrance,  CA  905  05 

John  R.  Rice 
Professor 

Computer  Science  Dept. 

Purdue  University 
Math-Sci.  428 
W.  Lafayette,  IN;  47907 

William  Riddle 
sda,  Inc. 

1670  Bfcar  Mtn.  Dr. 

Bhulder,  CO.  8C3C3 

Joseph  E.  Urban 
Computer  Science  Department 
Univ.  of  S.W.  Louisiana 
P.0.  B6x  44330 
Lafayette,  LA.  7(5  04: 

Jack  C.  Wileden 
Univ.  of  Mass. 

115  Pelham  Road 
Amherst,  MA.  01002 


410  DOCUMENTS  PRODUCED 


The  following  documents  are  products  of  the  Task  Force: 

o  STARS  Joint  Task  Force  Report  contains  a  brief  history:  of 
the  Task  Force,  references  to  the  other  documents  produced 
by  the  Task  Force,  and  an  appendix  on  the  Raleigh  workshop 
(this  document). 

o  STARS  Program  Strategy  gives  the  rationale  for  the  program 
and  outlines  its  implementation  and  organization. 

o  STARS  Program  Management  Plan  is  intended  to  become  the  for¬ 
mal  agreement  among  DoD  Components  covering  how  they  will 
work  together  to  plan  and  execute  the  program. 

o  STARS  Implementation  Approach  describes  the  approach  to  com¬ 
posing  implementation  tasks  and  the  acquisition  approach  for 
constructing  automated  support  environments. 

o  STARS  Functional  Task  Area  Strategies  are  eight  documents 
one  for  each  of  the  functional  task  areas.  They: state  over¬ 
views,  objectives,  strategy ;  and  tasks  for  each  area.  They 
are: 

-  STARS  Functional  Task  Area  Strategy  for  Measurement 

STARS  Functional  Task  Area  Strategy  for  Human  Resources 

STARS  Functional  Task  Area  Strategy  for  Project  Manage¬ 
ment 

-  STARS  Functional  Task  Area  Strategy  for  Systems 

-  STARS  Functional  Task  Area  Strategy  for  Application 
Specific 

-  STARS  Functional  Task  Area  Strategy  for  Acquisition 

-  STARS  Functional  Task  Area  Strategy  for  Human  Engineer¬ 
ing 

-  STARS  Functional  Task  Area  Strategy  for  Support  Systems. 

o  A  Candidate  Strategy  for  the  Software  Engineering  Institute 
describes  a  possible  plan  and  organization  for  the  Software 
Engineering  Institute.  The  options  for  the  Software 


5.0  EARLY  CRITICAL  PATH  TASKS 


The  Task  Force  has  identified  the  following  tasks  as  being  early 
tasks  on  critical  paths.  These  are  foundation  tasks  which  the  Task 
Force  recommends  be  initiated  expeditiously;  These  are  detailed 
further  and  their  rationale  discussed  in  the  STARS  Implementation 
Approach  and  the  Functional  Task  Area  Strategies. 

o  Construction  of  Support  Environments  (STARS  Implementation 
Approach) 

o  Establish  baseline (s)  (Measurement  Task  Area) 

o  Determine  program- success  measures  (Measurement) 

o  Establish  measurement  criteria,  metrics,  and  experimental 

techniques  for  each  task  (Measurement) 

o  Develop  tools  and  techniques  for  instrumentation  and  data 
analysis  (Measurement) 

o  Perform  hunan  resource  technical  and  managerial  skill 
assessment  (Human  Resources) 

o  Identify : important  application  areas  (Application  Specific) 

o  Form  user  groups  taking  advantage  of  existing  groups 

-  end-user  groups  (Application  Specific) 

-  developnent /support  groups  (Support  Systems) 

o  Develop  evaluation  criteria  for  Ada  and  computer  systems 

architectures  (Systems) 

o  Provide  Ada  access  to  target  run-time  system  (Systems) 

o  Develop  system  reliability : enhancement  techniques  and  tools 
(Systems) 

o  Review  impediments  in  current  acquisition  practices 
(Acquisition) 

o  Establish  Acquisition  Panel  (Acquisition) 
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o  Establish  approach  to  protection  of  software  including 

proprietary,  classification,  and  foreign  export  issues 
(Acquisition) 

o  Establish  mechanism  to  evaluate  and  prioritize  human 

engineering  research,  methodology,  and  tools  (Human 
Engineering) 

o  Conduct  Methodman3  experiment  (Support  Systems) 

o  Prepare  to  evaluate  tools,  environments,  and  methods,  par¬ 
ticularly  environment  definition  and  design  evaluation  cri¬ 
teria  for  use  at  decision  points  (Support  Systems,  STARS 
Implementation  Plan.) 

o  Develop  tool  integration  concepts,  techniques,  and  tools 
(Support  Systems) 

o  Perform  functional  analysis  of  project  management  (Project 
Management ) 

o  Perform  R&D  on  alternative  paradigms  or  revolutionary: 
approaches  (Support  Systems,  Systems,  and  potentially : else¬ 
where  ) . 


3p.  Freeman  and  A.  I.  Wasserman,  Comparing  Software  Design  Methods 
for  Ada:  A  Study  Plan.  Ada  Joint  Program  Office,  November  1982. 
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1.4  Manley  Panel  Findings 


APPENDIX  1.1 


Workshop  Program 


DoD  SOFTWARE  INITIATIVE  WORKSHOP 


Raleigh  Marriott 
Raleigh ,  North  Carolina 

7-9  February  1983 


General  Chairperson 


‘Larry  E.  Druffel,  Lt.  Colonel,  U5AF 
ODUSORE  (RiAT) 


Program  Co-Chairpersons  -Samuel  T.  Redwine,  Jr. 

Mitre  Corporation  and 

William  E.  Riddle 

software  design  &  analysis,  inc. 


Local  Arrangements  Chairperson  -James  B.  Clary 

Research  Triangle  Institute 


AGE  N  D  A 


MONDAY  -  7  February  1983: 

10:00  a.m.  -  12:15  p.m.  General  Session 

-Opening  Remarks 
L.  Druffel 

-welcome  to  the  Research  Triangle  Area 
George  R.  Herbert,  President  of  RT1 
-The  Software  Initiative  Effort  in 
the  DoD  Research  and  Advanced 
Technology  Context 
H.  Mark  Grove,  Assistant  Deputy 
Under  Secretary  Defense  for 
Research  and  Engineering 
(Research  and  Advanced  Technology) 

-Software  Initiative  Background  and 
Objectives 
L.  Druffel 


-Overview  of  the  Software  Initiative 
Technical  Plan 
S.  Redwine,  W.  Riddle, 

12:30  p.m.  -  1:30  p.ro.  -LUNCH 


17 


Parallel  Sessions 


1:30  pn.  -  5:30  p.m. 


3:00  p.m  -  3:30  p.r.. 


6:30  p.m.  -  8:00  p.m. 


TUESDAY  -  8  February  1983: 


8:00  a.m.  -  11:00  p.m. 


9:15  a.m.  -  9:45  a.m. 


-1)  Support  Systems  Panel 
Co-Chairpersons:  George  Sumrall 

Ann  Marmor- Squires 
Vice-Chairperson:  Jade  Wileden 

2)  Human  Resources  Panel 
Chairperson:  Charles  Oglesby 
Vice-Chairperson:  Joseph  Urban 

3)  Acquisition  Panel 
Co-Qiairpersons:  D.  Burton  Newlin 

Bemie  Zampolich 

Vice-Chairperson:  Joseph  Beardwood,  III 


BREAK 


-Reception  with  Cash  Bar 


Parallel  Sessions 


-1)  Systems  Panel 
Chairperson:  Stephen  Squires 
Vice-Chairperson:  Geoffrey  Prank 

2)  Human  Engineering  Panel 
Chairperson:  Carol  Morgan 
Vice-Chairperson:  Elizabeth  Kruesi 

3)  Project  Management  Panel 
Chairperson:  H.  0.  Lubbes 
Vice-Qiairperson:  Donald  Reifer 

-BREAK 


11:15  a.m.  -  12:15  p.m. 
12:30  p.m.  -  1:30  p.m. 
1:45  p.m.  -  5:45  p.m. 


3:45  p.m.  -  4:15  p.m. 


-LUNCH 

Continuation  of  Morning  Sessions 

-1)  Application  Specific  Panel 
Chairperson:  Paul  Cohen 
Vice-Chairperson:  John  R.  Rice 

2)  Technology  Insertion  Panel 
Chairperson:  Harold  Falk 
Vice-Chairperson:  Joe  Pox 

3)  Measurement  Panel 
Chairperson:  Samual  DiNitto 
Vice-Chairperson:  Janet  Dunham 

-BREAK 
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WEDNESDAY  9  February  1983: 

Parallel  Sessions 


8:30  a.Rt.  -  9:30  a.m.  -1)  Support  Systens  Panel 

2)  Human  Resources  Panel 

3)  Acquisition  Panel 

10:00  a.m.  -  11:00  a.m.  -1)  Systens  Panel 

2)  Hunan  Engineering  Panel 

3)  Project  Management  Panel 

11:15  a.m.  -  12:15  p.m.  -1)  Application  Specific  Panel 

2)  Technology  Insertion  Panel 

3)  Measurement  Panel 

9:30  a.m.  -  10:00  a.m.  -BREAK 


12:30  p.m.  -  1:30  p.m.  -LUNCH 


General  Session 

1:30  p.m.  -  4:30  p.m.  -Summary  of  Task  Area  Plans  and 

Closing  Remarks 

L.  Druffel,  S.  Redwine,  W.  Riddle 


APPENDIX  1.2 


Summary  of  Issues  Raised  at  Workshop 


APPENDIX  1.2 


SUMMARY  OF  ISSUES  RAISED  AT  WORKSHOP 


1.0  INTRODUCTION 

Each  functional  task  area  panel  compiled  its  own  list  of  issues 
raised  at  the  workshop.  They  vary  among  the  panels  in  style  and  for¬ 
mat,  but  all  are  presented  here  exactly  as  they  were  prepared. 

2.0  MEASUREMENT  ISSUES 

1.  Emphasis  on  measurement  is  needed  throughout  the  whole  life  cycle 
Requirements  and  design  measures  can  locate  serious  problems  while 
they  are  still  inexpensive  to  fix.  Measurements  for  and  in  the 
testing  phase  can  help  insure  reliability,  simplify  testing,  and 
provide  an  indication  of  when  testing  is  complete.  Metrics  and 
measurements  related  to  the  maintainability  of  software  products 

and  the  measurement  of  the  maintenance  process  itself  have  a  very 
high  leverage  toward  holding  down  costs  because  that  phase  can  be 
70Z  of  the  total  costs.  Finally,  user-oriented  measures  and 
measures  of  user  performance  should  be  included. 

The  panel  concurred,  and  all  of  the  above  will  be  included  in 
the  measurement  task. 

2.  Several  issues  and  concerns  were  raised  with  regard  to  data 
collection: 

(a)  How  much  data  is  adequate  to  validate/calibrate  the 
metrics,  and  establish  the  baselines? 

(b)  How  does  one  inforce  the  anonymity  of  the  data? 

(c)  How  does  one  collect  a  superset  of  data  to  support 
future  measures,  metrics,  and  baselines? 

(d)  How  does  one  insure  generality  of  the  data  so 
that  it  will  have  wide  application  but  not 

be  too  burdensome  and  costly  to  collect? 

(e)  How  does  one  insure  the  integrity  of  the  data? 

(f)  The  number  of  environments  automatically  instrumented* 
should  be  larger  than  that  supported  (developed)  by 
the  STARS  Pro  gran . 


Issue  (a),  was  not  completely  resolved,  but  a  representative  set 
of  embedded  systems  modules  will  be  sought  to  provide  a  veil  rounded 
data  base.  At  a  cost  of  between  5  and  121  overhead  in  systems  under 
development  or  maintenance,  the  data  collection  to  support  the 
activities  must  be  limited. 

Issue  (b),  had  no  resolution  since  some  of  the  data  necessary 
would  leave  little  doubt  as  to  the  source. 

Issue  (c),  had  no  resolution,  although  some  of  the  members  of 
the  audience  felt  it  was  possible.  The  panel  generally  disagreed 
with  the  supposition. 

Issue  (d),  had  no  immediate  resolution,  but  will  be  given  emphasis 
in  the  initiative.  The  panel  felt  it  was  resolvable. 

On  issue  (e),  proposals  were  made  to  use  IYSV  and/or  the  contractors 
own  Q/A  to  insure  the  integrity  of  the  data.  The  panel  felt  that 
relying  on  IV&V  would  not  help  to  integrate  measurement  into  the 
development  process.  The  panel  and  audience  agreed  that  the  program 
manager  and  developer  teams  must  recognise  the  value  of  the  data  and 
use  it  themselves  if  its  integrity  was  to  be  valid. 

On  issue  (f),  the  panel  did  not  think  resources  vere  available  to 
support  instrumentation  of  more  than  one  environment  to  automatically 
collect  and  analyze  the  detailed  data  needed.  It  is  intended  to 
provide  a  stand-alone  system  to  work  with  uninstrumented  environments, 
but  these  would  not  be  able  to  collect  all  the  data  needed.  It  was 
strongly  emphasized  that  there  would  be  two  levels  of  data  collected, 
one  at  the  very  detailed  level,  and  one  at  the  higher  level,  which  is 
already  collected  on  most  projects  (costs,  lines  of  code,  etc.).  The 
latter  would  generally  be  available  for  free  if  anonymity  could  be 
insured. 

3.  A  recurring  theme  concerned  the  problems  associated  with  the  lack 
of  commonality  of  definitions  for  terms  like  "line  of  code"  and 
"error."  While  recognizing  this  as  a  problem  that  would  be  solved 

by  the  measurement  task,  no  solutions  vere  proposed  by  the  panel  or 
from  the  floor. 

4.  An  issue  was  raised  on  how  to  get  the  data  collection  and 
metrics  implemented  as  an  integral  part  of  a  contract.  Several 
good  suggestions  vere  made:  the  metrics  and  their  collection 
should  be  tied  to  the  Work  Breakdown  Structure  of  a  contract. 

A  menu  of  metrics  should  be  proposed  for  a  user,  rather  than  a  /►. 

dictated  set.  Strong  educational  support  should  be  provided. 

Circulate  a  list,  of  metrics  and  data  this  task  area  will  collect,  to 
a  vide  audience  for  comment  and  refinement. 

2.0  HUMAN  RESOURCES  ISSUES 
1.  Coordination  betveen  tasks  is  needed. 
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2.  Focus  should  also  include  training  the  emerging  personal  computer  population 
and  secondary  schools  to  utilize  proper  techniques. 

3.  Develop  other  necessary  skills  along  with  the  professional 
develop  activities. 

4.  Use  learning  aids  to  help  transfer  Ada  computer /software 
education  and  training  to  non-ADP  types. 

5.  Current  exchange  programs  not  fully  utilized. 

6.  Standard  curriculum  for  software  engineering. 

7.  Starting  salary  structures  fo  S.E.'s  need  revision. 

8. *  Who  should  oversee  the  Human  Hesources  and  STARS  tasks? 

9.  Emphasis  should  also  be  placed  on  retraining  efforts. 

*  No.  8  unresolved.  Other  7  have  been  addressed  or  can 

be  specifically  incorporated  into  a  detailed  plan  later. 

3.0  PROJECT  MANAGEMENT  ISSUES  AND  DISPOSITIONS 


1. 

2. 

3. 

4. 

5. 


ISSUE: 

DISPOSITION: 

ISSUE: 

DISPOSITION: 

ISSUE: 

DISPOSITION: 

ISSUE: 

DISPOSITION: 

ISSUE: 

DISPOSITION: 


Exceaaive  fo.us  on  tools. 

Tools  will  be  supportive  of  management  concepts. 

Early  user  involvement. 

A  users  group  will  be  formed  to  guide  tool  development. 

Definition  of  Acquisition  Manager,  Project  Manager,  etc. 

The  problem  of  titles  for  people  who  perform  the  project 
management  function  has  been  dealt  with  in  the  plan. 

Application  to  software  maintenance. 

The  plan  has  been  written  to  specifically  include 
the  maintenance,  support,  or  redevelopment  issues. 

Needs  clear  statment  of  problem  and  who  will  be  helped. 

The  plan  has  a  clear  statement  of  the  problem  and  a 
discussion  of  who  is  to  be  helped. 


6.  ISSUE:  Need  close  coupling  to  acquisition,  measurement,  and 

other  areas. 

DISPOSITION:  The  differences  between  the  Project  Management  and 

Acquisition  task  areas  have  been  detined  and  section 
3  of  the  plan  identifies  the  interfaces  with  the 
other  task  areas. 

7.  ISSUE:  Prioritize  tasks. 


23 


DISPOSITION : 

To  be  accomplished. 

8. 

ISSUE: 

JLC  interface. 

DISPOSITION: 

To  be  identified. 

9. 

ISSUE: 

Measurement  and  validation. 

DISPOSITION: 

The  plan  identifies  validation  as  part  of  the  tasks 

Measures  of  effectiveness  will  be  developed  as  part 

of  the  Measuranent  Task  Area. 

% 

10. 

ISSUE: 

Out-year  research. 

DISPOSITION: 

Out-year  research  has  been  identified  and  clarified 

by  the  plan. 

11.  ISSUE:  Elimination  of  the  IPMTS. 

DISPOSITION:  The  IPMTS  has  been  retained  as  a  necessary  part  of 
the  tool  evaluation  and  as  a  means  of  refining  and 
validating  the  Project  Management  Functional 
Analysis. 

12.  ISSUE:  Original  plan  identified  an  intelligent  work  station 

as  a  possible  implementation  of  an  IPMTS.  This  view 
vas  questioned  as  being  too  limited. 

DISPOSITION:  The  physical  implementation  strategy  has  been 
eliminated  from  the  plan. 

13.  ISSUE:  Hosting/portability  problems  vith  the  IPMTS. 

DISPOSITION:  The  IPMTS  is  considered  to  be  a  prototype  and  is 
useful  as  a  means  of  identifying  useful  tools, 
validating  concepts  and  as  a  baseline  for  the 
Advanced  Tool  Set.  Acquisition  strategies  for 
tvo  IPMTS s  have  been  identified  which  minimize 
the  hosting  and  probability  issues. 

14.  ISSUE:  Tool  distribution,  maintenance  support. 

DISPOSITION:  Results  from  the  tool  set  efforts  will  be  fed 
to  the  the  Support  Systems  Task  Area  for 
integration/ interface  vith  the  support 
environment.  Distribution  and  maintenance 
will  be  handled  by  the  Software  Engineering 
Institute. 

15.  ISSUES:  Leasing,  liability  and  proprietary  rights. 

DISPOSITION:  These  problems  have  been  identified  as  issues  which 
need  to  be  addressed  by  the  Acquisition  Panel. 
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16.  ISSUE: 


Forecast  next  generation  problems. 


DISPOSITION: 

17.  ISSUE: 
DISPOSITION: 

18.  ISSUE: 
DISPOSITION: 

19.  ISSUE: 
DISPOSITION: 

20.  ISSUE: 
DISPOSITION: 

21.  ISSUE: 
DISPOSITION: 

22.  ISSUE: 
DISPOSITION: 

23.  ISSUE: 
DISPOSITION : 

24.  ISSUE: 
DISPOSITION: 

25.  ISSUE: 
DISPOSITION : 

26.  ISSUE: 


The  strategy  of  the  plan  ia  auch  that  it  can  adapt 
to  the  next  generation  problems. 

Create  a  generic  work  breakdown  structure  (VBS)  model. 

The  capability  for  a  generic  VBS  model  is  incorporated 
in  the  notion  of  acquisition  models  and  policies  and 
procedures . 

Impact  on  project  management  of  software  engineering 
technology. 

The  plan  identifies  a  strong  relationship  of  software 
engineering  to  project  management. 

Explore  Government/contractor  relationship. 

The  plan  explores  this  relationship  and  identifies 
issues  to  be  addressed  by  the  Acquisition  Panel. 

Employ  case  study  analysis. 

The  plan  employs  case  studies  to  validate  the 
results  of  the  Project  Management  Functional 
Analysis  Task. 

Ada  transition  strategy. 

The  plan  identifies  the  requirement  for  the  tools 
to  be  integrated  or  interfaced  with  an  APSE-like 
environment. 

Impl mentation  decision  point  for  tool  set. 

Implementation  of  deliverable  tool  sets  is 
covered  in  the  support  systems  and  software 
engineering  institute  plans. 

Advisory  council  participation. 

Some  as  #2. 

Expert  system/management  synergy. 

The  use  of  Imovledge  base  and  artificial  intelligence 
concepts  are  embedded  in  the  planning  for  an  Advanced 
Project  Management  Tool  Set. 

Plan  should  include  leadership  training. 

Included  in  the  plan. 

Question  of  efficacy  of  a  management  simulator  as  a 
training  aid. 
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DISPOSITION:  The  management  simulator  has  been  included  in  the 
plan  baaed  on  the  success  that  the  Naval  War 
College  has  had  using  digital  simulations  and 
gaming  techniques  to  emphasize  tactical  decision 
making  and  planning. 


4.0  SYSTEMS  ISSUES 
Issues 

o  Starting  vitb  Ada  does  not  help  people  with  existing  systems 

o  Need  more  comprehensive  target  system  support  as  early  as 
possible  including  use  of  special  devices  and  connection  to 
VHSIC  and  VLSI 

o  Need  to  bring  VHSIC  and  VLSI  design  to  point  vhere  it  may  be 
more  easily  used  as  part  of  system  development  effort  as 
needed 

o  Need  more  support  for  cross  development 

o  Need  more  support  for  existing  target  systems 

o  Should  target  system  software  be  in  systems  or  support  sys- 
terns  area? 

o  Suggestions  received  for  tasks  and  ongoing  or  planned 
activities  to  support 

Response*  bv  Systems  Panel 

o  Focused  on  scope,  strategy,  relation  to  other  areas 

o  Recognized  enormous  size  and  complexity  of  the  systems  space 

o  Identified  system  properties  of  interest 

o  Recognized  the  need  for  a  market  interface  as  model,  con¬ 
straints  ,  evaluation 

o  Recognized  the  need  for  limitations  in  state-of-art  to  be 
identified 


o  Recognized  quantum  model  of  properties 


Recognised  need  Co  bsve  more  integrated  view  of  system  pro¬ 
perties 

Recognized  scale  of  problem  and  need  for  scalable  results 

Formulated  high  leverage  market  model  with  technology  con¬ 
straints 


5.0  APPLICATION-SPECIFIC  ISSUES 

1.  PROPRIETARY  SOFTWARE  RIGHTS  ISSUE  —  What  mechanisms  and/or 
strategies  are  needed  to  protect  the  proprietary  interests  of 
contractors?  Will  contractors  be  willing  to  divest  themselves 
of  proprietary  rights?  Is  there  a  danger  that  after  donating 
software  to  the  Government  or  putting  such  software  in  the 
public  domain  that  the  developer  will  be  barred  from 
commercially  marketing  this  software  because  of  it  becoming 
classified? 

2.  USER  GROUP  ISSUE  ~  How  do  we  promote  the  formation  of  user 
groups?  What  should  be  the  Government  role?  Should  industry 
organizations  be  encouraged  to  take  the  lead?  Can  existing 
groups  be  used  for  the  nuclei? 

3.  CATEGORIZATION  OF  APPLICATION  AREAS  —  What  scheme  should  we 
use  to  categorize  application  areas? 

4.  CHOICE  OF  APPLICATION  AREAS  —  What  application  areas  are 
ripe  for  immediate  infusion  of  funds?  What  are  the  areas  to 
consider  for  the  more  advanced  technologies  (VHLL,  etc.)? 

Should  we  aim  for  any  short  term  goals? 

5.  FUNDING  ISSUE  —  How  do  we  promote  leverage  from  existing  DoD 
progrmns?  Should  there  be  a  policy  for  joint  STARS/Component 
funding  of  projects  in  the  Application-Specific  Task  area? 

6.0  ACQUISITION  ISSUES 

1.  ISSUE:  In  general  we  acquire  systems  not  software. 

RESOLUTION:  Incorporated  in  plan. 

2.  ISSUE:  Taking  software  out  of  a  system  (5000.29) 

frequently  makes  it  impossible  to  reinsert  it. 

RESOLUTION:  Really  a  Project  Management  issue  -  dismissed. 

3.  ISSUE:  Problems  often  begin  with  unrealistic  development 

schedules. 

RESOLUTION:  Input  data  for  Task  1.  Also  a  Project  Management  issue 

4.  ISSUE:  Problems  also  begin  with  a  poor  hardware /software 

mix. 

RESOLUTION:  Input  data  for  Task  1.  Also  a  Project  Management  issue 

5.  ISSUE:  Contracting  vehicles  have  been  developed  on  the 

basis  of  bardware(  and  need  modification  to  properly 
address  software  issues.  . 

RESOLUTION:  Input  Data  for  Task  1. 
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.  ISSUE: 

RESOLUTION : 

7.  ISSUE: 

RESOLUTION: 

8.  ISSUE: 


RESOLUTION: 

9.  ISSUE: 

RESOLUTION: 

10.  ISSUE: 

RESOLUTION: 

11.  ISSUE: 
RESOLUTION: 

12.  ISSUE: 

RESOLUTION: 

13.  ISSUE: 


RESOLUTION: 

14.  ISSUE: 

RESOLUTION: 

15.  ISSUE: 

RESOLUTION: 

16.  ISSUE: 
RESOLUTION: 


Much  more  dollars  and  time  aust  be  spent  at  the 
front  end  of  prograss  to  prevent  downstrema  problems . 

Noted  in  Rian  but  really  a  Project  Management  issue. 

Additionally,  life  cycle  issues  must  be  addressed  in 
this  front  end  process. 

Incorporated  in  Plan. 

It  Bust  be  recognized  that  software,  from  development 
through  06M,  is  an  evolving  process  with  both 
incremental  (ECPs)  and  revolutionary  changes  brought 
about  by  rapidly  changing  technology. 

Incorporated  in  Plan. 

The  ECPs  above  often  lead  to  an  adversary  relationship 
between  government  program  managers  and  contracting 
officers. 

Incorporated  in  Plan.  Also  a  Human  Resources  Issue. 

Firm  Fixed  Price  contracts  become  viable  only  when 
the  product  is  testable,  and  will  not  change. 

Input  data  for  Task  1. 

Problem  with  software  testing  vice  system  testing. 
Input  data  for  Task  1. 

Software  should  be  considered  in  the  DSARC/ASARC 
process. 

Really  a  Project  Management  issue. 

There  must  be  a  great  deal  sure  interaction  between 
government  and  industry  to  streamline  acquisition, 
increase  productivity,  reliability  and  reusability 
of  software  products. 

Incorporated  in  Plan. 

This  leads  naturally  to  discussion  of  government 
and  industry  rights  in  data,  a  very  difficult  problem* 

Incorporated  in  Plan. 

There  is  a  need  for  a  standard  work  breakdown 
structure  for  software  development. 

Incorporated  in  Plan. 

I  • 

There  is  a  need  for  better  DIDs. 

Input  data  for  Task  1. 
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17.  ISSUE:  There  it  a  great  need  for  uniforait;  in  application 

of  Policiea  and  Regulations  (even  vithin  a  single 
service). 

RESOLUTION:  Incorporated  in  Plan. 

18.  ISSUE:  Unless  ve  get  coapatibility  between  AIE  and  ALS  we 

will  have  lover  productivity,  and  delay  IR&D 
leverage  from  industry. 

RESOLUTION:  Input  data  for  Task  1.  Also  a  PM  issue. 

19.  ISSUE:  There  is  great  concern  over  testing. 

RESOLUTION:  Noted. 

20.  ISSUE:  At  the  same  time,  ve  have  to  protect  the  dollars 

for  testing  and  quality  assurance. 

RESOLUTION:  Really  a  Project  Management  issue. 

21.  ISSUE:  1V&V,  is  it  worth  it,  and  hov  to  do  it  incrementally? 

RESOLUTION:  Incorporated  in  Plan. 

22.  ISSUE:  Need  a  risk  sharing  (Industry/DoD)  mechanism. 

RESOLUTION:  Incorporated  in  Plan. 

23.  ISSUE:  Can  ve  use  weighted  guidelines  to  encourage  both 

productivity  and  the  use  of  IR&D  funds  for  software 
related  developments. 

RESOLUTION:  Incorporated  in  Plan. 

24.  ISSUE:  Ve  should  make  use  of  fast  prototypes  (recognized 

as  throw  avays)  for  competitive  flyoffs,  between 
competing  vendors,  to  speed  the  development  process. 

RESOLUTION:  Incorporated  in  Plan. 

25.  ISSUE:  Ve  should  concentrate  more  on  immediate  problems 

as  opposed  to  long  term  issues. 

RESOLUTION:  Both  issues  are  addressed  in  the  Plan. 

26.  ISSUE:  More  than  50Z  of  software  acquired  by  DoD  are  major 

system  updates.  Too  often  these  updates  are 
acquired  through  an  attenuated  process  which  eliminates 
many  of  the  safeguards  applied  to  initial  system 
acquisitions.  This  often  leads  to  big  problems. 

RESOLUTION:  Incorporated  in  Plan. 

27.  ISSUE:  A  great  deal  more  could  be  done  vithin  the  existing 

policies  and  regulations  than  is  generally  done  today. 
This  is  because  of  a  lack  of  understanding  of  these 
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policies  end  regulations. 

RESOLUTION :  Incorporated  in  ?lan.  Also  is  a  Hunan  Resources  issue. 


7.0  HUMAN  ENGINEERING  ISSUES 

1.  Several  people  pointed  out  that  the  plan  was  entirely  concerned 
with  the  human-computer  interface  instead  of  with  the  human 
engineering  of  the  entire  process  of  software  development  and  support. 
This  widening  of  the  scope  of  human  engineering  has  been  incorporated 
into  the  latest  revision  of  the  plan. 

2.  One  person  suggested  that  we  consider  the  user's  conceptual 
model  of  the  system.  This  was  considered  a  worthy  research  topic 
with  a  longer-term  payoff. 

3.  One  person  pointed  to  the  problem  of  the  long  time-lag  in 
applying  research  results.  This  problem  cuts  across  the  entire 
Initiative.  Ve  believe  that  this  problem  can  be  lessened  by 
initiating  a  focused  research  program  that  is  directed  at 
solving  the  specific  set  of  problems  falling  within  the  realm 
of  the  Initiative. 

4.  Several  people  reminded  us  that  most  embedded  systems  do  not 
have  a  CRT  interface  yet  the  plan  seemed  to  be  directed  towards 

a  terminal  interface.  It  was  suggested  that  we  focus  on  other 
types  of  I/O  such  as  voice,  tactile,  and  analog  displays.  It  was 
agreed  that  this  was  under-emphasized  in  the  original  plan.  The 
focus  of  the  plan  has  now  shifted  to  included  the  end-user  of 
embedded  systems. 

5.  One  person  commented  on  the  importance  of  measurement  to  the 
goals  of  the  Human  Engineering  Task  Area.  Ve  need  ways  of 
obtaining  feedback  from  the  field  use  of  end  products.  The 
support  environment  will  be  instrumented  as  part  of  the  Measurement 
Task  Area.  The  problems  involved  in  obtaining  feedback  about  the 
use  of  embedded  systems  will  be  addressed  by  Subtask  4  of  the 
current  plan. 

6.  Ve  were  reminded  of  the  severity  of  the  consequences  of  poor 
human  engineering  of  tactical  embedded  systems.  This  has  been 
mentioned  in  the  current  revision. 

7.  Ve  received  one  comment  about  the  need  for  validating  the 
human  engineering  methodology.  Ve  interpreted  this  in  two  ways, 
both  of  vbich  ve  agree  are  important  and  necessary.  Ve  need 

to  apply  the  methodology  and  then  collect  data  to  shov  that  the 
system  is  actually  better  as  a  result  of  having  been  developed 
under  such  a  methodology.  Ve  also  need  to  work  out  mechanisms 
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either  through  acquisition,  management,  or  the  uae  of  tool*  to 
ensure  adherence  to  the  methodology.  Subtask  4  of  the  current 
plan  is  directly  concerned  with  the  need  to  collect  data.  The 
necessary  linkages  must  be  set  up  with  the  Management, 
Acquisition,  and  Support  Systems  Task  Area  to  ensure  adherence 
to  the  methodology. 

8.  One  person  commented  on  the  lack  of  a  clear  responsibility 
for  the  measurement  aspect  of  Human  Engineering.  It  vas  unclear 
whether  it  should  lie  with  the  Measurement  Task  Area  or  with 
Human  Engineering.  These  responsibilities  were  clarified  in 
the  current  plan  by  assigning  that  responsibility  to  Human 
Engineering  with  support  from  the  Measurement  Task. 

9.  The  point  was  made  that  prototypes  can  be  very  useful  in 
determining  system  requirements.  It  is  assumed  that  the  use 
of  prototypes  belongs  under  Subtask  1  of  this  plan.  It  also 
seas  to  overlap  with  the  Support  Systems  Task  Area. 

10.  One  person  suggested  that  Human  Engineering  should  export 
its  expertise  in  evaluation  and  experimentation  into  the  other 
task  areas.  Ve  recognize  that  stany  of  the  people  involved  in 
human  factors  activities  are  trained  in  experimental  design 
and  statistical  analysis.  This  is,  however,  the  responsibility 
of  the  Measuraent  Task  Area  although  a  synergistic  link  between 
the  two  areas  is  certainly  expected. 

11.  The  panel  felt  that  there  is  a  clear  need  for  a  steering 
group  to  be  responsible  for  the  focus  of  the  methodological 
activities.  This  includes  assessing  the  currently  available 
techniques  and  guiding  the  selection  of  further  activities. 

In  the  current  revision,  these  functions  have  been  incorporated 
into  the  previously  planned  Research  Advisory  Panel.  The 
establishment  of  this  panel  is  now  a  part  of  Subtask  1 
(Methodology  Development). 

12.  There  were  several  issues  concerning  the  human  engineering 
of  the  support  environment.  The  panel  noted  that  there  is 
essentially  no  work  on  the  human  engineering  of  automated 
environments  for  software  development.  There  is  much  talk 
about  human  engineering  vhich  focuses  on  discussions  of  the  .use 
of  graphics,  mice,  and  other  devices.  Ho  one  appears  to 
address  the  basic  principles  of  interface  design  or  systematic 
experimentation,  both  of  which  fall  within  the  domain  of  a 
true  human  engineering  discipline.  Automated  support 
environments  present  special  problems  for  human  engineering. 

The  plan  has  been  revised  to  include  a  discussion  of  the  need 
to  address  the  issue  of  maintaining  a  consistent  user  interface 
across  tools  while  allowing  for  portability  of  tools  across 
environments. 
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8.0  SUPPORT  SYSTEMS  ISSUES:  SUMMARY  OF  WORKSHOP  SESSIONS 

The  Support  Systens  panel  first  set  in  open  session  on  Monday 
afternoon,  7  February  1983.  Over  300  people  attended  this  session. 
After  brief  introductory  remarks  and  introduction  of  the  panel 
members,  a  20-minute  overview  of  the  task  area  vas  given.  Questions 
from  the  audience  were  encouraged  at  this  point.  Further  details  of 
the  Support  Systems  technical  activities  plan  including  milestone 
charts  and  manpower  estimates  were  then  presented  in  approximately 
two  hours.  Numerous  issues  were  raised  by  the  audience  during  this 
latter  presentation;  individuals  were  asked  by  the  panel  chairpersons 
to  write  down  their  questions  and  comments. 

This  first  open  session  resulted  in  57  verbal  responses  and  110 
written  responses  from  the  audience.  Some  of  the  written  responses 
recorded  the  verbal  questions  raised  during  the  session.  During  a 
closed  panel  session  on  Tuesday,  the  responses  were  read  by  the  panel 
members  and  condensed  into  approximately  31  issues.  These  31  issues 
fell  into  six  major  areas  of  concern  which  will  be  discussed  in  the 
next  paragraphs.  The  panel  also  received  written  comments  about 
additional  on-going  or  planned  projects  that  were  supportive  of  the 
Initiative's  goals,  particularly  in  the  Support  Systems  task  ares. 

In  addition  to  specific  issues  raised  about  the  Support  Systems 
task  area,  several  global  issues  were  raised  by  the  audience.  These 
questions  addressed  the  relationship  of  this  task  area  to  the  others 
in  the  Initiative,  the  role  of  public  review  and  comment  in  the  Ini¬ 
tiative,  the  relationship  of  the  Initiative  to  Ada  and  its  associated 
activities,  the  assumptions  made  about  other  research  and  development 
activities  and  other  major  initiatives/progrons  that  are  underway  or 
planned,  the  impact  of  the  proprietary  vs.  public  domain  issues,  and 
the  role  of  the  marketplace.  Some  of  the  issues  were  addressed  ,in 
later  open  sessions;  others  will  be  addressed  in  revisions  of  the 
plan. 
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The  audience  raised  several  questions  about  the  apparent  lack  of 
explicit  goals  and  objectives  in  the  Support  Systems  task  area  plan 
as  presented  in  the  open  session.  Although  there  vere  goals  and 
objectives  presented  and  a  rationale  for  the  tasks,  the  presentation 
did  not  explicitly  state  them  at  the  outset.  This  issue  vas  resolved 
in  the  second  open  session;  the  overall  goal  of  the  task  area  vas 
presented  and  specific  objectives  vere  stated  explicitly.  This  is 
also  being  incorporated  into  the  current  revision  of  the  plan. 

Several  questions  indicated  that  the  plan  seemed  to  not  be 
encouraging  innovation  and  not  advancing  technology.  There  vere 
related  questions  on  the  impact  of  the  Support  Systems  task  area  on 
more  innovative  vays  for  DoD  to  develop  systems  and  a  need  for  an 
early  identification  of  key  research  issues  to  be  addressed.  This 
issue  vas  addressed  in  the  second  open  session,  kesearch  is  an  on¬ 
going  activity  in  the  plan  but  vas  not  made  visible;  there  vill  be 
k&D  pursued  along  both  evolutionary  and  revolutionary  paths.  The 
research  aspects  of  the  plan  vill  be  more  visible  in  the  current 
revision  ot  the  plan. 

There  vere  several  questions  regarding  the  role  of  knovledge- 
based  systems  technology  in  the  task  area  plan.  Some  questioners 
felt  that  there  vas  inadequate  emphasis  on  that  technology.  This 
issue  vas  resolved.  One  of  the  revolutionary  thrusts  of  the  k&d  in 
this  task  area  vill  focus  on  a  know ledge-based  systems  paradigm. 
This  vill  be  incorporated  into  the  current  revision  of  the  plan. 

There  vere  numerous  questions  about  environments  and  methodolo¬ 
gies.  The  environment  questions  focused  on:  (1)  vas  the  plan  to 
develop  single  or  multiple  environments;  (2)  the  characteristics  of 
the  environments  and  the  categories  of  users  that  vould  be  supported; 
(31  a  generic  environment  or  application-specific  environments;  (4)  a 
"model"  environment;  (3)  the  role  of  the  environment  to  enforce1  or- 
support  the  methodology;  (6)  hov  to  evolve  the  environment  and  re- 


34 


engineer  and  integrate  tools  into  it;  (7)  how  to  deal  with  existing 
tools  and  existing  software.  The  methodology  questions  focused  on: 
(1)  v as  the  plan  to  develop  single  or  multiple  methodologies;  (2) 
support  of  the  entire  life-cycle;  (3)  methodology  for  building 
environments  and  integrating  tools.  Some  of  the  issues  raised  were 
addressed  during  the  second  open  session.  There  was  also  significant 
discussion  related  to  these  issues  during  the  closed  panel  sessions. 
The  current  revision  of  the  plan  will  incorporate  the  results  of 
these  comments  and  discussions.  These  issues  could  not  all  be 
resolved  in  the  short  time  at  the  Workshop. 

Several  questions  were  raised  about  the  levels  of  effort  pro¬ 
posed  for  the  tasks.  It  was  unclear  to  the  audience  whether  the  man¬ 
power  estimates  referred  to  management  of  the  tasks  or  actually  car¬ 
rying  out  the  tasks  on  contract.  Some  of  the  audience  felt  that  the 
levels  of  effort  were  extremely  low.  There  was  concern  that  without 
leveraging  on  industry  and  the  marketplace  it  would  be  extremely  dif¬ 
ficult  to  carry  out  the  Support  Systems  task  area  with  the  manpower 
estimates  presented.  In  the  technology  experimentation  and  demons¬ 
tration  tasks,  there  was  particular  concern  that  the  magnitude  of  the 
problem  was  extremely  underestimated.  There  were  also  several 
related  questions  about  the  assumptions  being  made  about  activities 
outside  the  Initiative  and  whether  that  influenced  the  manpower  esti¬ 
mates.  This  issue  still  needs  further  work. 

Several  questions  were  raised  about  the  specific  sequencing  and 
choice  of  tasks  to  be  carried  out.  There  was  concern  that  the  paral¬ 
lelism  of  the  tasks  as  presented  was  not  realistic.  Other  comments 
indicated  that  certain  tasks  that  need  to  be  carried  out  appeared  to 
be  missing  from  the  plan.  Some  of  the  comments  were  res*,  ved 
quickly;  however,  this  issue  still  needs  further  work. 
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9.0  SOFTWARE  ENGINEERING  INSTITUTE  ISSUES  AND  ALTERNATIVES 


The  following  12  trees  received  considerable  attencion  by  Panel 
■embers  and  were  addressed  in  open  session  comments  at  the  Workshop. 
A  brief  description  of  the  issue  and  alternatives  is  given  and  a 
rationale  provided  for  the  Panel  consensus. 

o  Should  the  Software  Engineering  Institute  (SEI)  focus  its 
software  engineering  efforts  on  the  embedded  computer  appli¬ 
cations  area  to  a  broader  scope  of  software  development 
areas  ? 

Consensus :  Embedded  computer  systems. 

Rationale:  DoD  embedded  computer  systems  (ECS)  are  critical 
to  our  National  defense  posture.  They  stress  the  extremes 
of  such  system  characteristics.  Unlike  the  ADF  application 
environment,  where  industries  focus  their  R&D  efforts,  lit¬ 
tle  industry  effort  is  put  on  defense  unique  ECS  applica¬ 
tions.  The  DoD  must  focus  its  attention  on  this  area. 
Industry  R&D  will  satisfy  most  Defense  ADP  needs. 

o  Should  the  SEI  provide  products  and  services  and  focus  its 
attention  the  Defense  community  or  to  all  areas  of  users  of 
ECS? 

Consensus :  The  Defense  community. 

Rationale:  This  question  is  related  to  the  first.  Because 
DoD  resources  are  limited,  the  SEI  should  focus  on  defense 
needs.  Non-sensitive  spinoff  technology  will  have  commer¬ 
cial  application  and  will  be  made  available. 

The  scope  of  the  SEI  mission  and  focus  could  be 
broadened,  perhaps,  with  a  larger  basis  of  support  from  out¬ 
side  the  DoD.  This  is  discussed  in  #8  below. 

o  Should  the  SEI's  primary  mission  be  limited  to  engineering 
and  integration  of  software  development  technology  or  should 
it  support  and  accomplish  software  research  as  well? 

Consensus :  The  consensus  favored  including  research  in  the 
SEI  mission.  ' 

Rationale:  A  significant  portion  of  the  SEI  staff  is  to  be 
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tude  up  of  visiting  researchers  vho  will  bring  software 
research  products  to  the  Institute  for  engineering  and 
integration  into  the  Institute's  software  development 
environment.  A  strong  research  progrms  will  encourage  this 
infusion  of  quality  people. 

In  addition,  the  Institute  should  do  research  on 
software  development  environment  measurements  and  technology 
transition. 

o  Should  the  Institute  be  responsible  for  maintaining  the  DoD 
"standard”  software  development  environment  for  all  service 
to  use? 

Consensus :  No. 

Rationale:  Current  Service  policies  separately  address  the 
standardization  of  computer  resources  including  development 
systems.  As  the  Ada  language  and  programming  support 
environments  are  developed,  there  will  be  a  convergence 
towards  commonality. 

The  SEI  will  develop  and  maintain  an  advanced  environ¬ 
ment  compatible  with  the  Ada  programming  environments 
developed  by  the  Services.  This  will  assure  the  efficient 
transition  of  new  tools  from  the  Institute  to  the  Services 
and  their  contractors. 

o  Should  the  Institute  provide  software  engineering  education 
and  training?  A  possible  extension  of  the  mission  might  be 
an  extensive  academic  program,  to  the  possible  extent  of  a 
degree  granting  institution. 

Consensus :  Education  and  training  to  support  tool  and  prac¬ 
tice  transition  should  be  accomplished.  Additional  educa¬ 
tion  in  general  software  development  involving  the  SEI 
environment  should  be  carried  on.  The  training  should  be 
provided  to  key  Service  and  contractor  personnel  vho  would 
then  provide  the  education  to  larger  groups  of  environment 
users. 

Rationale:  Broadening  the  scope  of  education  would  require 
greater  SEI  resources  than  available.  The  Services  have 
their  undergraduate  and  graduate  programs  as  well  as  other 
training  programs  which  should  provide  for  general  needs., 
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Should  Che  SEI  provide  facilities  and  offer  computer  pro¬ 
cessing  (including  softvare  tools)  resources  and  services 
for  fee? 

Consensus :  It  seened  appropriate  that,  at  the  beginning, 
only  limited  services-f or-f ee  should  be  provided.  Such  ser¬ 
vices  involving  the  use  of  nev  tools  could  be  provided  as  a 
vay  of  introducing  these  tools  aB  veil  as  evaluating  their 
effectiveness  in  real  applications. 

Rationale:  A  larger  operation  to  provide  complete  environ¬ 
ment  and  remote  computer  service  capability  for  the  DoD  com¬ 
munity  seemed  very  ambitious.  It  could  be  an  SEI  grovth 
possibility  if  its  effectiveness  and  feasibility  is  demon¬ 
strated  by  initial,  smaller  scale  experiments. 

Are  the  scope,  size,  and  budget,  as  proposed,  compatible? 

Consensus :  The  size  and  scope  are  compatible.  The  budget  as 
proposed  vas  not  adequate. 

Rationale:  The  budgeted  amounts  for  personnel  were  too 
small.  (These  amounts  have  been  increased  in  the  current 
plan.) 

Should  the  management  and  the  support  for  the  SEI  be  DoS 
based  or  broader  based?  Should  other  government  agencies, 
i.e.,  NASA,  FAA,  etc.,  be  included?  Should  organizations 
outside  Government  be  involved  in  support  and  management  of 
the  SEI? 

Consensus :  Although  strong  suggestions  for  a  "National"  SEI 
were  heard,  the  Fanel  recommended  limiting  management  to  the 
DoD.  Support  grants  can  be  accepted  from  outsiders  but  not 
direction. 

Rationale:  The  focus  should  be  maintained  to  support  the 
Defense  community  embedded  computer  softvare  engineering 
area.  Thus,  management  should  be  limited  to  the  DoD. 

Bov  should  the  SEI  be  managed  vithin  the  DoD  —  by  OSD,  by 
the  Services  jointly,  or  by  a  single  Service? 

Consensus :  The  SET  is  a  part  of  the  STARS  Progran  and  prob¬ 
ably  remain  part  of  this  program  as  long  as  it  exists.  The 
STARS  Progran  is  currently  planned  as  a  tri-service  managed 
progran,  the  management  to  be  composed  of  Service  represen- 
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Rationale :  The  above  was  not  thoroughly  worked  out.  Strong 
Service  views  exist  that  the  SEI,  and  possible  the  STARS 
program  as  well,  should  be  Service  managed. 

What  vehicle  or  host  institution  should  be  used  to  organize 
and  establish  the  SEI  —  a  university  or  university  consor¬ 
tium,  a  not-for-profit  company,  a  for-profit  corporation,  or 
a  Service  or  Agency? 

Consensus ;  The  university,  university  consortium,  or  not- 
for-profit  corporation  were  favored. 

Rationale :  The  DoD  is  personnel  resource  limited.  Estab¬ 
lishing  the  SEI  within  the  DoD  would  cause  a  personnel 
resource  redistribution  which  would  adversely  effect  other 
functions.  DoD  salary  structure  would  limit  the  effective¬ 
ness  in  obtaining  quality  personnel. 

The  motivation  of  for-profit  industry  appears  incompa¬ 
tible  with  the  free  interchange  of  ideas  necessary  for  the 
SEI. 


Some  expressed  the  belief  that  researchers  would  be 
better  attracted  to  a  university  environment. 

Orientation  of  the  Institute  —  user  needs  or  technology 
push? 

Consensus :  User  needs. 

Rationale:  The  requirements  of  software  developers  in  indus¬ 
try  and  in  Service  centers  must  be  the  driver  for  the  SEI 
activities. 

Type  of  personnel  required  for  the  SEI  mission  —  world- 
class  researchers  or  other  types? 

Consensus :  A  variety  of  personnel  will  be  required  to  staff 
a  successful  SEI,  both  engineering  and  research. 

Rationale:  A  specific  type  of  engineering  resource  is 
required  to  transition,  engineer,  and  integrate  software 
engineering  tools  and  develop  advanced  environments.  1  A 
thorough  understanding  of  the  application  —  software 
engineering  of  real  systems  —  is  needed.  "World-class” 
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1.0  INTRODUCTION 


Faced  with  a  serious  threat  of  being  overtaken  by  massive* 
nat  i  ona 1 ly-sponsor ed  software  technology  initiatives  in  other 
countries  and  a  consequent  erosion  of  U.S.  superiority  in  mis- 
s  i  on-cr  i  t  i  cal  ,  software-dependent  defense  systems*  the  DoD  is 
establishing  a  Software  Intiative*  Software  Technology  for 
Adaptable*  Reliable  Systems  (STARS)*  to  ensure  that  the  pres¬ 
ent  U.S.  lead  is  maintained. 

In  October  1982  a  draft  of  the  Initiative's  program  plan  was 
issued.  In  February  1983  this  plan*  augmented  by  work  to  date* 
was  publicly  reviewed  at  the  DoD  Software  Initiative  Workshop 
in  Raleigh*  NC*  under  the  chairmanship  of  LtC  Larry  Druffel. 
"Special  Panel  2>"  the  STARS  program  review  panel*  was  convened 
to  evaluate  the  draft  plan  independently  of  the  other  Workshop 
panels.  This  report  documents  the  findings  and  recommen¬ 
dations  of  Special  Panel  2. 

Membership  included:  John  Manley  (ITT)*  Chairman*  Barry  Boehm 
(TRW),  Neil  Eastmar  (IBM),  Donn  Philpot  (EE),  Terry  Straeter 
(GD),  A 1  Whittaker  (Honeywell)  and  Bill  Wulf  (Tartan  Labs). 

The  Panel  members  concur  unanimously  with  this  report.  There 
are  no  minority  or  dissenting  opinions. 


1.1  PURPOSE  AND  SCOPE 


Special  Panel  2  was  convened  by  the  Workshop  chairman*  LtC  Lar¬ 
ry  Druffel*  to  evaluate  the  STARS  program  strategy  documenta¬ 
tion  with  particular  emphasis  on  the  following  areas: 

•  purpose  of  STARS 

•  potential  benefits  to  DoD  missi  on-cr itical  systems 

•  program  del i verables 

•  advocacy  and  implementation  strategies 

•  economic  issues  and  the  composition  of  industry  involve¬ 
ment 

•  continuation  of  results 

•  leveraging  by  industry  of  Initiative-generated  advances 

The  STARS  strategic  plan  as  represented  by  current  documenta¬ 
tion  was  reviewed.  Panel  members  minimized  interaction  with 
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other  Workshop  participants  to  ensure  a  concentrated*  uninter 
rupted  review  and  to  ensure  that  the  plan  itself,  not  partic 
i pants  *  interpretations,  was  the  subject  of  discussion. 
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2.0  EXECUTIVE  SUMMARY 


The  Panel  strongly  concurs  that  a  Sof tware ' In 1 1 i at  5 ve  is  essen¬ 
tial  to  maintaining  leadership  in  DoD  mission-critical 
systems.  The  specific  needs  for  improvement  identified  in  the 
Initiative  documentation  appear  to  be  complete. 

However,  as  presently  structured  the  STARS  plan  will  fall  short 
of  its  goals.  The  paragraphs  below  summarize  recommendations 
to  realign  the  plan  towards  successful  attainment  of  its  goals. 

The  Panel  recommends  that  a  r es u 1 t s -b a s ed  goals  statement  be 
formulated  and  that  plan  elements  be  revised  to  concentrate 
upon  problems  whose  solutions  lead  to  results.  The  attendant 
objectives  should  give  prominent  priority  to  the  need  for  dras¬ 
tically  shortened  elapsed  times  between  software  concept  defi¬ 
nition  and  operational  deployment.  Goals  statements  in  the 
October  draft  document  are  broad  and  essentially 
technology-oriented;  all  technology  areas  germane  to  the  soft¬ 
ware  life  cycle  are  addressed  at  roughly  equal  levels  of 
emphasis.  Results-based  goals  and  problem-based  plans  are 
necessary  to  ensure  timely,  continuing  and  coordinated  pro¬ 
gress  towards  needed  capabilities  rather  than  just  increased 
technology  potential. 

The  Panel  recommends  that  the  revised  STARS  plan  assure  strong 
industry  participation  and  complementary  industry  investment. 
As  a  stand-alone  program,  presently  proposed  funding  levels 
are  inadequate  by  several  factors.  Without  energetic  industry 
participation,  effective  technology  insertion  and  reduction  to 
practice  are  not  achievable  within  time  objectives. 

The  Panel  recommends  that  a  STARS  acquisition  strategy  be 
estab 1 i shed  to : 

•  Facilitate  rapid  evolutionary  insertion  of  technology  into 
operational  mission-critical  systems 

•  Allow  identification  and  funding  of  very  high  potential 
technolog i es 

•  Secure  the  program  elements  necessary  for  successful 
implementation  support 

The  acquisition  strategy  should  call  for  both  "evolutionary" 
and  "revolut i onary"  deliverables.  Evolutionary  deliverables 
are  "showcase"  operational  software  elements  that  have  been 
produced  on  shorter  schedules  and  with  greater  functionality, 
quality  and  maintainability  than  previously  achieved,  together 
with  the  improved  techniques,  tools,  environments  and  compo¬ 
nents  that  make  the  showcase  achievements  routine. 
Revolutionary  deliverables  are  high-payoff,  high-risk  proto¬ 
type  and  brassboard  software  elements. 

53 


Executive  Summary 


The  Panel  recommends  that  the  role  of  the  Software  Engineering 
Institute  be  re-evaluated.  Its  mission  and  responsibilities 
should  be  sufficient  to  attract  top-caliber  software  business 
management  and  technical  management  talent.  The  role  and  com¬ 
position  of  the  Software  Engineering  Institute  in  providing 
essential  STARS  implementation  support  are  discussed  in  the 
body  of  this  report. 
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3.0  FINDINGS  AND  RECOMMENDATIONS  DISCUSSION 


3.1  STARS  GOAL:  RESTATEMENT  AND  RATIONALE 


The  STARS  goal  should  be  restated  as  follows: 

•  "The  STARS  goal  is  to  increase  the  current  U.S.  DoD  lead  in 
operational  software:  to  build  and  support  more  complex* 
higher  quality  mission-critical  systems  on  shorter  sched¬ 
ules  and  to  assure  that  leadership  is  maintained  through¬ 
out  the  1990's." 

The  restatement  shifts  the  emphasis  of  the  program  from  reduc¬ 
ing  software  people  and  dollar  requirements  to  accelerating 
the  deployment  of  increasingly  complex  systems.  That  is*  we 
agree  that  cost  reduction  and  productivity  in  general  are  nec¬ 
essary  to  the  future  of  the  United  States  but  believe  that  sys¬ 
tem  functionality  and  timeliness*  and  therefore  quality 
software*  should  be  the  primary  drivers  of  the  defense  software 
technology  base.  Accepting  this  restated  goal  helps  to  pre¬ 
clude  the  many  productivity  solutions  available  through 
combining  program  resources  and  extending  schedules.  It 
recognizes  that  overcoming  the  significant  barriers  to  com¬ 
pressing  schedules  which  exist  today  while  simultaneously 
increasing  delivered  system  function  will  oive  the  United 
State  clear  and  continued  leadership  in  software. 

From  the  restated  goal  it  follows  that  the  plan  should  be 
restructured  to  assure  strong  industry  participation*  not  only 
in  the  sense  of  contract  opportunities  but  just  as  importantly 
in  the  sense  of  attracting  industry  investment.  Without  such 
investment  the  proposed  DoD  funding  would  need  to  be  increased 
by  several  factors  for  the  Initiative  to  meet  its  stated  goals. 
Even  if  sufficient  funds  were  available  such  an  approach  would 
be  undesirable*  since  those  for  whom  technology  transition  is 
intended  (industry)  would  tend  to  resist  and  lag  if  they  were 
not  spontaneous  part i c i pants . 

In  order  to  engage  industry  there  must  be  economic  consider¬ 
ations  given  to  major  prime  contractors*  subcontractors  and 
entrepreneur i al  firms.  Technology  targets  must  be  of  common 
concern  across  DoD  and  where  possible  to  the  commercial  sector. 
The  software  acquisition  process  must  complement  profitability 
end  protect  trade  secrets. 
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3.2  STARS  PLAN  DOCUMENTATION:  REORGANIZATION  AND  RATIONALE 


The  Panel  finds  the  organization  and  structure  of  the  present 
STARS  plan  to  give  cause  for  concern.  The  plan  is  primarily 
organized  by  a  categorization  of  technologies  applicable 
across  the  software  life  cycle.  Each  technology  category  is 
and  Mill  be  a  necessary  part  of  the  STARS  approach,  but  a  tech¬ 
nology-based  organization  does  not  provide  a  framework  for 
problems,  priorities  or  results.  A  fragmented  approach 
appears  to  be  probable  since  there  is  no  consistent  thread  of 
purpose  to  bind  the  task  areas.  A  technology-based  approach  as 
contrasted  with  a  problem-based  approach  will  cause  problems 
in  four  key  aspects: 

•  Advocacy 

Should  the  STARS  plan  fail  to  clearly  delineate  problem 
areas  and  objectives,  the  Services  will  not  be  able  to 
evaluate  program  elements  in  light  of  anticipated  capabil¬ 
ities,  threats  and  missions,  and  Congress  will  be  hindered 
in  their  policy  and  funding  deliberations.  STARS  may 
appear  as  s  large  expenditure  with  an  uncertain  return. 

•  Industry  support 

A  "shopping  list"  of  potential  technology  capabilities  is 
unlikely  by  itself  to  draw  active  industry  interest. 
Reduction  to  practice  and  integration  into  existing  prac¬ 
tices  are  frequently  expensive  obstacles;  clear  customer 
(DoD)  goals  and  plans  are  prerequisite  for  industry  moti¬ 
vation. 

•  Implementation 

Without  a  problem-  and  solution-based  organization,  an 
understanding  of  requirements  leading  to  complementary  and 
synergistic  task  definitions  is  unlikely  to  occur. 

•  Management 

A  fragmented,  non-pr i or i t i zed  program  does  not  set  a  base 
for  effective  or  efficient  management  of  scarce  resources, 
nor  for  optimal  decisions  in  the  case  of  alternative  oppor¬ 
tunities  or  d i scr et i onar y  resource  allocations. 

The  Panel  recommends  that  the  STARS  document  be  repackaged  into 
a  Plan  that  is  based  upon  Application  problem  areas  (discussed 
in  the  deliverables  strategy  rationale)  to  assure  linkage  and 
integration  of  the  necessarily  wide  range  of  technology  ele¬ 
ments  . 
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3.3  ACQUISITION  STRATEGY  FOR  MAJOR  DELIVERABLES 


The  Panel  recommends  an  acquisition  strategy  calling  for  STARS 
major  deliverables  in  three  categories: 

•  Implementation  support 

•  "Evolutionary"  products 

•  "Revolutionary"  products 

Implementation  support  deliverables  span  the  wide  range  of 
processi  practice!  standards*  data  and  measurement  areas 
essential  to  support  the  developing  state-of-the-art  software 
engineering  environment.  These  are  seen  as  a  principal  concern 
of  a  Software  Engineering  Institute*  discussed  in  a  later  sec¬ 
tion  of  this  report.  An  evolutionary  product  marks  the 
successful  reduction  to  standard  practice  of  an  improved  capa¬ 
bility*  a  revolutionary  product  is  a  prot  type  or  brassboard 
demonstration  of  high-payoff  but  high-risk  advanced 
capability.  In  both  cases*  the  products  should  represent  sol¬ 
utions  of  specific  problems  or  attainment  of  specific 
objectives . 


3.3.1  Deliverables  strategy  rationale 


DoD  operational  software  should  be  categorized  by  Application 
Areas  within  which  common  algorithms*  techniques*  components 
and  tools  may  reasonably  be  expected  to  apply.  Examples  of 
Application  Areas  are  C3»  avionics*  flight  control*  fire  con¬ 
trol*  electronic  warfare*  trainers  and  simulators  and  the 
like.  Each  Application  Area  should  then  be  analyzed  to  identi¬ 
fy*  for  each  phase  of  the  software  life  cycle*  the  development* 
support  and  operational  elements  which  are 
Application-specific  and  those  which  are  common  to  all  Appli¬ 
cations.  Contractors  may  then  propose  the  use  of  improved 
capabilities  in  operational  software  development  contracts*  or 
the  procuring  agent  may  solicit  such  use.  Upon  demonstration 
in  a  successful  deliverable  the  improved  capability  is  added  to 
growing  repertoires  of  ’standard’  Application  or 
cross-Application  capabilities. 

The  first  use  and  demonstration  of  new  or  improved  capabilities 
in  operational  software  procurements  under  the  funding  spon¬ 
sorship  of  STARS  will  significantly  accelerate  effective 
reduction  to  practice.  An  environment  in  which  all  the  expense 
and  risk  of  novelty  is  born  by  the  contractor  carries  powerful 
incentives  for  each  contractor  to  seek  to  be  second  -  and  never 
f i rst . 
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3.3.2  Evolutionary  products 


Evolutionary  products  should  be  in  the  form  of  operational 
software  which  is  produced  on  shorter  schedules*  with  higher 
quality  (including  maintainability)  and  providing  greater 
functionality  than  previously  achieved.  These  improvements 
will  be  the  demonstrated  benefits  of  improved  or  new  software 
techniques*  components*  environments  and  tools.  Integration* 
adaptation  and  first  application  of  these  elements  to  the  pro¬ 
duction  of  evolutionary  deliverables  should  be  underwritten  by 
STARS  contract  funds*  and  they  should  in  turn  be  deliverables 
as  their  capabilities  are  proved. 

Evolutionary  products  should  be  procured  via  RFP’s  whose 
Statements  of  Work  identify  candidate  capabilities  and  request 
bidders  to  include  plans  to  use*  demonstrate  and  deliver 
improved  capabilities.  Award  determinations  would  include 
evaluations  of  evolutionary-related  bid  response  parts.  As  in 
the  VHSIC  program*  each  Service  should  sponsor  a  number  of 
evolutionary  products. 


3.3.3  Revolutionary  products 


Revolutionary  products  may  occasionally  be  in  the  form  of  oper¬ 
ational  software  but  will  usually  be  prototype  or  brassboard 
components  built  to  demonstrate  novel  capabilities.  The 
source  of  these  capabilities  may  be  advanced  research  or  tech¬ 
nology  development  projects*  the  revolutionary  deliverable 
will  be  the  vehicle  for  scaling-up  to  pre-production  levels  and 
experience  in  a  production-equivalent  environment.  Capabili¬ 
ties  successfully  used  in  revoluti onary  deliverables  become 
candidates  for  validation  in  evolutionary  deliverables. 


3.4  SOFTWARE  ENGINEERING  INSTITUTE 


The  October  draft  strategy  assumes  that  other  DoD  organiza¬ 
tions  will  be  responsible  to  see  that  DoD  expertise  is  main¬ 
tained  in  their  areas  and  that  the  Initiative  will  provide 
funds  to  selected  DoD  or gan i zat i ons  to  execute  and  manage  con¬ 
tracts  to  support  the  Initiative.  The  draft  views  a  Software 
Engineering  lstitute  as  being  the  principal  engineering  organ¬ 
ization  for  creating  a  state-of-the-art  software  environment* 
and  consequently  as  the  main  point  of  supply  for  newly  engi¬ 
neered  and  integrated  capabilities*  documentation*  training 
and  user  assistance.  Its  suggested  role  is  to  bridge  the  "gap 
between  R&D  activities  that  demostrate  new  techniques  in  a  con- 
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strained  domain  and  the  exploitation  of  those  techniques  on 
real  systems  .  " 


The  Panel  is  concerned  by  the  suggested  Software  Engineering 
Institute  focus  for  several  reasons: 

•  Staff  and  funding  would  need  to  be  greater  by  several  fac¬ 
tors  than  proposed  levels  to  undertake  such  activities. 

•  OoD  assumption  of  a  principal  implementation  and  supplier 
role  would  severely  limit  industry  incentives  to  actively 
parti c i pate  . 

•  Responsibilities  for  planning!  coordination  and  manage¬ 
ment  of  activities  distributed  across  academia*  the  Gov¬ 
ernment  and  industry  are  not  sufficiently  considered. 

The  Panel  recommends  that  the  role  of  the  Software  Engineering 
Institute  be  re-evaluated.  STARS  program  planning!  coordi¬ 
nation  and  management  from  both  a  business  management  and  a 
technical  management  point  of  view  could  be  principal  respon¬ 
sibilities  of  an  Institute. 

The  Software  Engineering  Institute  could  be  assigned  specific 
near-term  responsibility  for  a  STARS  implementation  plan*  a 
deliverables  acquisition  plan  and  a  technology  insertion  plan. 
The  Panel's  recommended  acquisition  strategy  for  evolutionary 
and  revolut i onary  deliverables  (discussed  in  paragraphs  above) 
establishes  the  major  framework  within  which  program  activ¬ 
ities  should  be  developed. 

Technology  development  scopes  are  well  covered  in  the  present 
Initiative  draft  document.  As  the  plans  outlined  above  develop 
and  are  implemented  the  Software  Engineering  Institute  could 
ensure  that  appropriate  linkage  and  transition  activities  are 
identified  and  put  in  place. 


3.4.1  STARS  Implementation  Plan 


Implementing  a  revised  STARS  plan  requires  two  major  steps: 

•  Step  1 

—  Repackage  the  STARS  documentation 

—  Establish  a  STARS  acquisition  strategy 

—  Develop  an  implementation  plan  that  includes  a  clear 
definition  of  essential  roles  and  missions*  to  include 
those  assigned  to  the  Software  Engineering  Institute. 
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—  Implement  the  repackaged  plan*  using  the  acquisition 
strategy  established  in  Step  1.  This  will  involve 
implementing  the  major  portions  of  the  repackaged  plan 
in  parallel . 

The  STARS  acquisition  strategy  task  needs  to  define  STARS 
acquisition  roles  and  r espons i b i 1 i t i es  within  DoD  and  the  Ser¬ 
vices*  to  establish  criteria  for  the  awards  and  deliverables  to 
be  pursued  in  Step  2*  and  to  define  an  appropriate  set  of  candi¬ 
date  mission-critical  areas  (C5,  flight  control,  fire  control, 
EM*  etc.)  for  implementation  as  evolutionary  operational  pro¬ 
ducts  and  r e vo 1 ut i onar y  prototype  products. 


3.4.2  Implementation  Support  Activities 


The  STARS  program  will  create  a  large  number  of  powerful  meth¬ 
ods*  tools  and  components  which  must  be  assimilated  into  a  uni¬ 
fied,  APSE-based  structure  and  infused  throughout  all  portions 
of  DoD  and  industry.  This  will  probably  require  major  involve¬ 
ment  of  the  Software  Engineering  Institute  function.  Imple¬ 
mentation  support  tasks  include: 

•  measurement  (as  a  technology  and  to  show  STARS  progress) 

•  acquisition  (of  technologies) 

•  Software  engineering  processes  and  business  practices 

•  software  engineering  standards  and  controls 

•  technology  interface  management  (cataloging*  tracking) 

•  Information  transfer 

•  tools*  methods  and  components  repository 

•  data  base  administration 

•  linkages 

These  tasks  would  probably  be  best  carried  out  within  or  under 
the  close  direction  of  the  Software  Engineering  Institute. 
Once  the  expanded  scope  of  Institute  rcspons  i  b  i  1  i  t  i  es  ,is 
determined*  including  interfaces  with  other  i mportant  agents 
such  as  the  JLC*  an  integrated  plan  for  effecting  the  STARS 
implementation  support  functions  must  be  developed  and  put 
into  action. 
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3.4.3  Technology  Insertion 


The  essential  functions  of  i  np 1 ementat i on  support  and  technol¬ 
ogy  insertion  must  be  provided  by  an  organization  that  is  inde¬ 
pendent  of  factional  biases.  Technology  insertion  (not  just 
technology  development)  should  be  the  primary  goal  of  the  Soft- 
Hare  Engineering  Institute.  In  this  regard*  the  Panel  agrees 
Hith  the  present  plan’s  strategy  of  rotating  academic*  Govern¬ 
ment  and  industry  personnel  through  the  Institute  since  tech¬ 
nology  is  most  effectively  transferred  through  people.  The 
Institute  should  not  be  exclusively  a  research  organization 
but*  to  be  effective*  must  have  mission  and  scope  that  attract 
world-class  talent  in  each  of  its  areas  of  responsibility* 
including  as  a  top  priority  software  business  management  and 
technology  management  as  well  as  the  best  technical  experts 
available. 

Given  that  sets  of  advanced  methods*  tools  and  components  are 
delivered  by  contractors  as  outlined  in  the  deliverables 
acquisition  strategy  in  paragraphs  above*  the  technically  dif¬ 
ficult  work  that  remains  is: 

•  smooth  integration  of  the  parts  across  contractor  packages 

•  achieving  wide  and  uniform  usage 

The  first  task  is  a  major  effort  which  is  seen  by  the  Panel  as  a 
possible  function  of  a  redefined  Software  Engineering  Insti¬ 
tute.  The  second  task  will  be  accomplished  partly  by  strongly 
encouraging  use  in  the  RFP  provisions  for  large-scale  DoD  sys¬ 
tem  acquisitions  and  partly  by  technology  insertion  activities 
recommended  by  the  Institute. 

The  technology  insertion  challenge  is  key  to  the  entire  DoD 
Software  Initiative.  Reuse  of  the  developed  state-of-the-art 
technologies  must  not  be  left  to  chance  —  it  must  be  made  to 
happen ! 
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